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The Impact of ABS Program Participation on Long-Term 
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Introduction 


National and international studies such as the recent Survey 
of Adult Skills’ provide strong evidence of the need for and 
economic value of adult basic skills (ABS). A growing body 
of research indicates that there is a strong economic return 

to basic skills at all levels of education.’ Estimates have been 
made of the potential economic benefits that would accrue 
from increased. educational attainment and levels of basic 
skills.? There is little rigorous research, however, showing that 
participation in basic skills programs directly impacts the skill 
levels, educational attainment, or social and economic well- 
being of adults with low levels of education. Most research on 
adult literacy development looks only at short-term changes 
as students pass through single adult basic skills programs. 
Most studies use short follow-up intervals and consider only 
program participants, making it difficult to see longer-term 
patterns of program participation and persistence and to 
assess long-term impact of ABS program participation.* 


Although ABS program evaluation and accountability reports 
typically show small gains for program participants in test 
scores and other outcomes, these studies rarely include 
comparison groups of nonparticipants, and most studies 
that do include such controls have not found statistically 
significant ABS program impact.’ Research is needed that 
compares adult literacy development among program 
participants and nonparticipants across multiple contexts 
and over significant periods of time to provide a life-wide 
and lifelong perspective on adult literacy development and a 
better assessment of program impact on a range of outcome 
measures. 


The Longitudinal Study of Adult Learning (LSAL) is one such 
lifelong and life-wide study. LSAL randomly sampled about 
1,000 high school dropouts and followed them for nearly a 
decade from 1998-2007. LSAL followed both participants 
and nonparticipants in adult literacy programs, assessing their 
literacy skills and skill uses over long periods of time, along 
with changes in their social, educational, and economic status, 
offering a rich picture of adult literacy development.° 


This is the second of a series of Research Briefs that utilize 
LSAL data to examine long-term impacts of ABS program 
participation on a range of outcome measures. Each Brief 
looks at a different outcome. ‘The first Brief considers the 
long-term impact of participation on individuals’ earnings. 
‘This second Brief examines the impact of participation on 
their literacy proficiency. Subsequent Briefs will examine the 


impact of participation on GED attainment, engagement in 
postsecondary education, and voting in general elections (a 
measure of civic engagement). 


This Research Brief addresses the following research questions: 


@ What is the impact of participating in an ABS program 
on subsequent literacy proficiency? 


@ What is the temporal course of that impact? 


@ To what extent does GED attainment mediate the impact 
of participation on proficiency? 


LSAL Design and Methodology 


The overall design, methodology, population, 
and instrumentation of LSAL are described in 
detail elsewhere,’ and only essential details are 
summarized here. 


Population and Sample 


The study population for LSAL was defined as adults who at 
the start of the study in 1998: lived in the Portland (Oregon) 
metropolitan area; were ages 18—44; had not completed high 
school nor were enrolled in high school or college; and were 
proficient but not necessarily native speakers of English. 

This defined population is a major segment of the target 
population of ABS programs operated by community colleges 
and other organizations in Oregon and across the country. 
‘The sample was drawn through random digit dialing, with 
oversampling of current participants in ABS programs to 
ensure adequate numbers of both program participants and 
nonparticipants in the sampled “panel” of 934 adults, who 
then were followed from 1998-2007. At study onset, the 
LSAL population had an average age of 28 and was evenly 
divided among males and females, with one-third from 
minority groups and one-tenth from immigrant populations. 
Nearly one in three reported having a learning disability. 


Some of these defining characteristics of LSAL’s population 
changed over time. Everyone's age increased, of course, while 
some adults received GEDs and college degrees, experienced 
changes in their employment and family situations, or 

moved away from the Portland area. LSAL followed its panel 
members regardless of these and other changes, with about 90 
percent of the original panel retained in the study until data 
collection ended in 2007.’ 
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Interviews and Assessments 


LSAL conducted a series of six periodic interviews and skills 
assessments in respondents’ homes:!° 


Wave 1: 1998-1999 
Wave 2: 1999-2000 
Wave 3: 2000-2001 
Wave 4: 2002-2003 
Wave 5: 2004-2005 
Wave 6: 2006-2007 


Note that the spacing of successive interviews was one year 
between Waves 1, 2, and 3 and two years between Waves 3, 4, 
jana,” 


The initial interview gathered background information (e.g., 
demographics, family-of-origin characteristics, K-12 school 
history). The initial and each successive interview collected 
information about recent social, economic, and educational 
activities (e.g., participation in basic skill programs; 
postsecondary education and training; employment, 

job characteristics, and earnings; household and family 
composition; life goals and aspirations).'? 


Literacy proficiency was assessed in each wave using alternate 
forms of the Document Literacy Scale of the Test of Applied 
Literacy Skills (TALS) developed by the Educational Testing 
Service. TALS assesses the ability of adults to extract and 
process written information in a variety of everyday document 
formats, such as forms, maps, tables, text displays, labels, 
and so forth. TALS instruments are similar to those used 

in many major national and international surveys of adult 
literacy, including the recently conducted Program for the 
International Assessment of Adult Competencies (PIAAC).'° 
TALS measures proficiency on a 0-500 scale, with scores 
often reported in terms of five proficiency levels. 


Participation in Adult Basic Skills Programs 


In each interview, individuals were asked if they currently 
were participating in an adult basic skills program to improve 
their reading, writing, or math skills, or prepare for the GED 
Tests, or had done so within the preceding 12 months (asked 
in Wave 1) or since the time of the preceding interview 

(asked in Waves 2-6). Those who reported such participation 
were asked follow-up questions about timing, intensity, and 
duration of their participation. In the Wave 1 interview, they 
also were asked about their participation in such programs 
prior to 12 months before the first interview (back to the time 


they had dropped out of high school). These ABS program 
participation data were converted into variables for the 
number of hours of participation in each time period. 


Key Findings 


About two-thirds (68%) of the LSAL population had 
participated in an ABS program between the time of 

leaving high school and the end of LSAL in 2007. This 

is much higher than the usual reported percentage of the 
adult education target population that is served in a given 
program year. There are several reasons for LSAL’s higher 
participation percentage: (1) LSAL’s 68 percent figure includes 
any participation over a long period of time rather than for a 
single program year; (2) LSAL’s population excludes adults age 
45 and above, an age group usually included in official counts 
of the target population but one that rarely participates in 
programs; and (3) LSAU’s figure includes any participation 
rather than the 12-hours-per-year minimum typically 
required for inclusion in state and federal program reports. 


Participation patterns in LSAL were often complex and 
fragmented, with many adults having multiple episodes of 
participation at different times and in different programs 
across the years of the study.'4 Figure 1 shows the estimated 
percentage of the LSAL population that ever participated 
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Figure 1. Percent of LSAL population who ever 
participated in ABS programs (line) and median hours of 
program attendance for those who participated (bars), 
by LSAL wave. LSAL waves 1-6 are placed on a time axis 
to represent their temporal spacing. 
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in an ABS program through each given wave of the study 
(line graph), as well as the median total hours of program 
attendance accumulated by participants (bar graph).'* By 
the end of the study in 2007, over half (54%) of the LSAL 
population who had never participated in ABS programs 
when LSAL began had participated in ABS programs, 
accumulating a median of 74 hours of attendance between 
1998-2006. 


Figure 2 shows the distribution of the five literacy 
proficiency levels in the LSAL population at Wave 1 and 
the distribution of the proficiency levels in the comparable 
statewide subpopulation (in terms of age, education, and 
language ability) within the 1991 Oregon Literacy Survey.'® 
Approximately 12 percent of the LSAL population, as well 
as the comparable statewide population, were at the lowest 
proficiency level (Level 1) and about 35 percent were at 
Level 2. 


Level 5 
Level 4 
Level 3 


Level 2 


Level 1 


Portland 


Oregon 


Figure 2. Wave 1 Distribution of Document Literacy in the 
LSAL Population and Comparable (age 18-44, no high 
school diploma or GED, proficient but not necessarily 

native speaker of English) Oregon Statewide Population. 

Sources: Portland: Longitudinal Study of Adult Literacy, 
Wave 1 (1998); Oregon: Oregon Literacy Survey (1991). 


The mean Wave 1 proficiency of ABS program participants 
(274) was lower than that of nonparticipants (285). 


Impact of Program Participation on Literacy 
Development 


Previous studies examined changes in LSAL literacy 
proficiencies over time using growth curve modeling.'” Those 
studies found relatively little systematic variation and modest 
overall growth in proficiency over time. Although the earlier 
research examined effects of observed characteristics on the 
shape of individual literacy growth curves, the effects of time- 
varying predictors such as ABS program participation were 
not included in analyses. There is, of course, good reason 


to expect program impact on literacy growth, since basic 
skills improvement and preparation for the GED Tests are 
core goals of ABS programs. Given the complex patterns 

of program participation described above, there are many 
ways of defining and measuring participation, comparing 
literacy proficiency of participants and nonparticipants, and 
estimating the impact of ABS program participation on 
literacy growth. 


Because there was not a large enough sample of individuals 
with literacy proficiencies assessed before they had 
participated in an ABS program, it is not feasible to compare 
preparticipation and postparticipation proficiencies of 
participants and nonparticipants. Thus, difference-in- 
differences and treatment effects models used in the first 
Research Brief to assess impact of participation on earnings 
are not suitable for use here.'® Instead, the fixed effects (FE) 
panel regression models used to assess impact of participation 
on earnings will be used, with literacy proficiency as the 
outcome variable. 


Fixed Effects Panel Regressions 


Care must be taken in comparing literacy growth of ABS 
program participants and nonparticipants. Individuals in the 
target population self-selected in terms of participating in 
ABS programs, and there may be other important differences 
between the two groups as well. The effects of those other 
differences may be confounded with effects of participation; 
this often is termed selection bias in program evaluation 
literature.'? Some selection bias in LSAL could be due to 
differences in observable characteristics of participants and 
nonparticipants such as age, amount of education, race/ 
ethnicity, immigration status, and so on. 


Bias in LSAL also may occur as a result of unobserved 
individual characteristics (e.g., noncognitive attributes 

such as motivation and self-confidence), sometimes termed 
omitted variables bias. The analytical method used—fixed 
effects panel regression—eliminates bias due to all differences 
in time-invariant individual characteristics, both observed 
and unobserved. The panel regressions analyze variations in 
literacy over time within individuals in relation to observed 
time-varying predictor variables, including ABS program 
participation. This framework makes it possible to model the 
impact of participation on literacy as well as the time course 
of that impact. Effects of other time-varying variables such as 
GED attainment also can be examined. 


Panel regressions of literacy were conducted with fixed effects 
of individuals and fixed effects of the six LSAL time periods. 
By fixing the effects of both individuals and time periods, 
these panel regressions highlight the effects of observable 


Research Brief: The Impact of ABS Program Participation on Long-Term Literacy Growth 


time-varying variables, such as ABS participation, on literacy 
trajectories. To measure different aspects of the complex 
participation patterns in ABS programs, several time-varying 
participation variables were contrasted in various FE models. 
‘These participation variables are alike in equaling zero at time 
points before an individual participated and equaling zero at 
all time points for individuals who never participated. Added 
to the fixed effects of individuals and time periods in the 
panel regressions, these time-varying predictors provide sharp 
measures of the effects of program participation on literacy 
development. 


Table 1 shows four FE panel regression models of literacy. 
Each of the models, A through D, contains one or two ABS 
program participation variables as time-varying predictors, 
as well as the fixed effects of individuals and time periods 
that are not shown in the table. The table shows estimated 
regression coefficients for participation variables involved in 
each model, along with their standard errors and ¢-statistics. 
Asterisks on ¢-values indicate statistical significance. 


Elapsed time could be relevant if, for example, program 
impact on literacy continues to develop over time as 
individuals use new skills and gradually gain proficiency. We 
see in Model B that the YEARS variable, measuring elapsed 
time since onset of participation, does not have a statistically 
significant regression coefficient. There is, thus, no significant 
linear increase in literacy proficiency following onset of 
participation as measured by this model. 


Model C is similar to Model B, but sets the onset of 
participation to the time the individual had accumulated 100 
or more hours of attendance in ABS programs. YRSCUM 100 
measures elapsed time since onset of participation so 

defined. YRSCUM100 has a statistically significant 
regression coefficient, reflecting a significant linear growth 

of impact over time following onset of a sufficient amount 

of attendance. The estimated regression coefficient of 1.65 
for YRSCUM100 indicates that sufficient participation is 
associated with added proficiency gains of 1.65 points per 
year. Although this is a very modest change over a single 

year, the impact accumulates into a more 


Table 1. Summaries of Four (A-D) Fixed Effects 
Panel Regressions of Literacy 


Coeff. in Robust 

Model Participation Variables Panel Regr. Std. Err. 
A CUMHOURS .0120 .0055 
B YEARS -.4884 .7035 
Cc YRSCUM100 1.6491 .5636 
D YRSCUM100 1.3076 .5488 
CUMHOURS .0071 .0055 


* p<.05; ** p<.01. 


(Regressions contain one or two time-varying measures of participation, 
as shown, as well as fixed effects of individuals and time periods, 
which are omitted from the table. See text for explanation.) 


substantial effect over time. For example, 
individuals who participated for 100 or 
more hours in the year preceding Wave 1 
are expected to have gained an additional 
eight years of this proficiency growth 


through Wave 6, resulting in a gain of 8 
t x 1.65 = 13 points, which is about 0.3 
2.18* standard deviations of proficiency (for the 
entire LSAL population). Participation 
a impact as measured by this model had a 
2:93** small to moderate effect that took many 
years to fully develop. Thus, both total 
sd hours of participation and elapsed time 


since onset of participation are significant 
positive predictors of literacy. 


Model A examines the effect of CUMHOURS on literacy. 
CUMHOURS is the total hours of program participation 
an individual accumulated through each LSAL time point. 
CUMHOURS is zero at the beginning of LSAL and increases 
at each subsequent time point by the number of hours 

the individual attended an ABS program. For individuals 
who never participated, CUMHOURS remains at zero. 
‘The positive, statistically significant regression coefficient 

of CUMHOURS indicates that, within individual literacy 
trajectories, the more hours of participation accumulated by 
individuals through a given point in time, the higher their 
literacy proficiencies tend to be at that point in time. 


In contrast, Model B examines variation of literacy in relation 
to elapsed time since onset of an individual’s participation. 


Model D includes both of these variables to determine 
whether they are distinct predictors of literacy and which 

is the stronger predictor. In Model D, only YRSCUM100 

has a statistically significant regression coefficient. 
CUMHOURS—a significant predictor in Model A—is no 
longer a significant predictor of literacy once effects of elapsed 
time following participation are taken into account. 


Figure 3 on page 5 illustrates the growth of literacy over 
time for ABS program participants and nonparticipants. 
Participants are defined in this figure as individuals who 
attended ABS programs for 100 or more hours, and 
nonparticipants are defined as those who did not.” 
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Concurrent Effects of GED Attainment 


290 
Since elapsed time after participation onset is 285 
a positive predictor of literacy proficiency, it is S 280 
important to examine what may be happening 2 275 
during the elapsed time period to drive proficiency %S 270 
upward. Receipt of a GED credential, for example, O65 
could mediate the impact of participation by Gai 
serving as a gateway to postsecondary education, = 
ie oh 255 
training, or other learning opportunities. 
250 


This can be examined by adding observable, 
time-varying measures of GED attainment to 

the FE panel regression models. The Appendix 
summarizes results for models in which time- 
varying measures of GED attainment are added to 
the basic FE model that uses YRSCUM100 as an 


=<=@— Participants 


eff Nonparticipants 


Figure 3. Mean Assessed Literacy Proficiency at Six LSAL Waves for ABS 
Program Participants. Participants are defined as individuals who attended 
an ABS program for at least 100 hours. (See text for explanation.) LSAL 
waves 1-6 are placed on a time axis to represent their temporal spacing. 


1 2 3 4 5 6 
LSAL Wave 


ABS participation measure. 


Results of the modeling described in the Appendix indicate 
that both ABS program participation and receipt of the GED 
credential have impact on literacy development. The impacts 
of program participation and GED attainment have different 
temporal shapes. Program participation has a slowly growing 
and long-lived impact on literacy development, whereas GED 
attainment appears to have a short-lived, pulse-shaped impact. 


Discussion 


The fixed effects panel regressions summarized in this 
Research Brief indicate that individuals who participate in 
ABS programs tend to have higher levels of future literacy 
proficiency as a result of participating. Their proficiency 
premiums are larger with more intensive participation. 
Minimal levels of participation, however, may not produce 
significant premiums. These findings hold when time-varying 
measures of GED attainment are controlled. 


This statistically significant but modest literacy premium takes 
time (on the order of years) to develop after participation. 
Because of the complexity of program participation patterns 
observed, LSAUs relatively small sample size limits the 
precision with which estimates can be made of how many 
hours of attendance or how long a follow-up period are 
required to see a significant literacy premium of a given size. 
Details vary with the measure of participation used. It is clear 
from the LSAL analyses conducted that higher intensities of 
participation (with a threshold of around 100 hours) have 
substantial impact on future literacy proficiency, an impact 
that typically takes years to develop after participation. 


To illustrate this impact, consider the participants who 
attended ABS programs for 100 or more hours during the 


year preceding their Wave 1 literacy assessments. Their literacy 
proficiencies averaged 260 in Wave 1 and 275 in Wave 6. 

This is an average gain of 15 proficiency points over the eight- 
year interval between Waves 1 and 6, or approximately 1.88 
proficiency points per year (note that this value is very close 
to the predictions of the YRSCUM100 model—Model C in 
Table 1). 


GED attainment does not seem to mediate the long-term 
impact of participation on literacy but does have a short- 
lasting positive impact on assessed proficiency around the 
time of credential receipt. This “pulse” impact of GED 
attainment on proficiency may reflect the “brushing up” of 


skills or test-taking ability. 


‘There are some important methodological limitations in 

this assessment of program impact on literacy development, 
especially when compared with the assessment of 
participation impact on earnings in the previous Research 
Brief. Although the fixed effects panel regression methods 
used. here make visible some of the dynamics of the impact 
of participation on literacy changes over time, no converging 
propensity score matching methods could be used with 

the data available to contrast literacy development in 
reasonably well-matched groups of program participants and 
nonparticipants. Therefore, a corresponding degree of caution 
is in order. 


Additional research with larger longitudinal data sets and 
those drawn from other contexts can help clarify some of 
these important details. The impact models developed here 
could address these questions more precisely if applied to 
larger longitudinal data sets that follow comparable ABS 
program participants and nonparticipants. 
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Future Research Briefs in the series look more closely at the 
impact of participation on GED attainment, postsecondary 
engagement, and voting (considered a measure of civic 
participation). 
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These data are for individuals who were assessed at all six 
waves. 


Of the LSAL population, 27 percent received a GED at 
some point during the study. Of those who received a 
GED, 76 percent had participated in an ABS program 
(37% of GED recipients had participated 100 or more 
hours during a 12-month period). 
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Appendix: Supplementary Information and Tables 


The impact of attaining a GED credential can be examined 
by adding observable, time-varying measures of GED 
attainment to the FE panel regression models. Table Al 
summarizes findings for models that add various time- 
varying measures of GED attainment to a basic FE model 
that uses YRSCUM100 as an ABS participation measure. 


The variable GED in Model E is defined to equal one 

in the year in which the GED is received and zero at all 
other times. This model thus adds a relatively short-lived 
“pulse” of extra proficiency at the wave in which the GED 
is received. As the table shows, the regression coefficient 
for this pulse is statistically significant, indicating that 
proficiency scores are elevated for a short period of time 
around GED attainment, added to the linearly increasing 
YRSCUM 100 impact of ABS program participation that is 
shown in the table to remain statistically significant. One 
way of understanding this relatively short-lived proficiency 
pulse is as a “brushing up” of skills 


Model G estimates the impact of GED attainment 
considered as a growth process by defining the variable 
YEARSGED as the number of years since the individual 
received the GED credential. YEARSGED equals zero 
before GED attainment and thereafter counts elapsed years 
since receipt of the credential; YEARSGED equals zero at 
all times for individuals who never receive the credential. 
Although YRSCUM100 continues to be a significant, 
positive predictor of literacy in Model G, YEARSGED does 
not have a significant regression coefficient. 


Thus, both ABS program participation and receipt of the 
GED credential have impact on literacy development. 
Their impacts have different temporal shapes. Program 
participation has a slowly growing and long-lived impact 
on literacy development. GED attainment has a short-lived, 
pulse-shaped impact on literacy development. 


associated with preparation for taking the 
GED Tests. 


Model F estimates the impact of GED 
attainment as a step function rather 
than a pulse. The HAVEGED variable 


Table A1. Summaries of Three (E-G) Fixed Effects 


Panel Regressions of Literacy 


(Each regression contains the two time-varying variables shown in the table as 
well as the fixed effects of individuals and time periods, which are omitted from 


the table. See text for explanation.) 


in Model F has value zero until the Goorain RObUeE 

individual receives a GED and value one Model Participation Variables Panel Regr. Std. Err. t 
thereafter; for individuals who never - YRSCUM100 ‘7946 Gece 3.10** 
attain a GED, HAVEGED always equals GED 7.6188 3.4153 2.23* 
zero. Although YRSCUM100 continues 

to be a significant, positive predictor of F pei 00 ra See 
literacy in Model F, HAVEGED does not 
have a significant regression coefficient. - YRSCUM100 1.6884 0.5644 2.99** 
This indicates that the literacy premium YEARSGED -0.2754 0.7058 -0.39 


accompanying credential receipt is short- 


* p<.05; ** p<.01. 
lived as measured in this model. 


